ci(changeset-gate): a level-axis reading that did not happen no longer concludes success, and the PR body is re-read on edited - #16897
Conversation
…evel verdict now fails, and the body is re-read on edit
`check-changeset-no-major.mjs`'s level axis exited 0 for both "judged and passed"
and "had nothing to judge". A check run concludes `success` or `failure` and has
no third word, so the two published the same conclusion on every surface that
reads conclusions rather than step logs, while `LEVEL AXIS: NOT MEASURED` stayed
in the job log.
`NOT MEASURED` is now split by whether the missing reading could have changed the
answer:
* not-measured-moot — no changeset grades `patch` a package whose
`packages/*/src/**` the diff moves, so `yes` and `no`
reach the same verdict. Still exit 0, and it now says
why it is green rather than printing a bare tick.
* not-measured-material — a `patch` sits on a package the diff grew, so the
declaration is the difference between `clean` and
`enforce`, and it was not readable. Exit 1, naming the
changeset, the package and both declarations.
The explicit `Clause-②: no` stays a pass on the very tree the unread reading
refuses: the opt-out is a declaration, never a tolerance.
Measured over the 45 most recent merges (31 judged, 14 exempt by label): exactly
one flips. Its body declares the clause inside a sentence rather than on a line,
so the gate read nothing and concluded success over an `@objectstack/spec` patch
beside a 124-line move in that package's own src.
Two lanes stay green because they are not pull requests at all — the RC cut
(`cut-rc.yml`, `workflow_dispatch`, no `--event`) and a local run reach
`no-pull-request`. On a real `pull_request` run an unreadable payload is a failure
instead: the runner writes that file, and a gate that could not read the input it
was owed has verified nothing.
The workflow half is the other half of the same fix. The remedy for the new
refusal is a PR-BODY edit, and a `pull_request` payload is a snapshot that
`rerun_failed_jobs` replays, so with no `edited` trigger the red could not be
cleared without pushing a commit. `pr-automation.yml` now subscribes to `edited`,
as this repo's two other PR-body-scoped blocking checks already do. `Check PR
Size` and `Auto Label` are excluded from it by the sentence they already carry
for label events, so the added volume buys exactly one job: measured
2026-09-01..09-08 from this repo's own run counts, about 557 `edited` events over
8 days against this workflow's own 3684, roughly +15%, at ~46s per job.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
ACCEPT — PR #16897, reviewed against GitHub and the tree, ⛔ not against the reportWhat I re-drove myself
⭐ Why this clears the bar I set in the claimThe brief said: whatever you choose, the distinction must reach the layer that gates a merge, and ⛔ "a careful reviewer could see it" is not an argument. Three things make this the right route rather than the convenient one:
⭐ And the discovery that ⭐ Two guards I want on the record
The blast radius, and what I am NOT claimingThe report judged the 45 most recent merges with the real script against their real bodies and labels: 31 judged, 14 exempt, 25 already declaring, 6 #16361 — fold-or-serial, confirmed⛔ Not folded, and the answer is right. This PR changes the verdict/exit machine and the trigger list; #16361 is about which packages the Open question, routed not answeredShould The cheaper route, correctly declinedRequiring the Checks30 checks, 0 failures, 0 pending. Flipping and arming now. Generated by Claude Code |
Part of #16776 — the two defects the card carries. It does NOT close the card: this PR lands the gate half, and the card's second direction for defect B (the declaration written at PR-creation time) reaches
.claude/agents/os-dev.md, a governed surface no seat here may land. See "What is deliberately NOT here".⛔ Not addressed here: 16361, the same gate's opposite direction (a false red from predicate scope). Untouched by this diff, held by the PM seat behind this PR.
The defect, re-driven before it was fixed
The card's Boundaries block says every reading in it is second-hand and was not re-driven by triage. It was re-driven here, on this branch's base, with the real script and real
--eventpayloads, on the very commit the card names (273247e56f, PR 16342's pre-fix head —@objectstack/spec: patchand@objectstack/runtime: patchbeside a diff that moves both packages'src/**).--eventpayloadClause-②: yesin the body, no carrierClause-②: probably— nonsense controlClause-②: nocb5d140d70, one word:minor), neither carrier--eventat all (the RC-cut / local lane)pull_requestrun whose payload will not parseThe firing control is what makes row 3 readable: rows 1 and 2 refuse the byte-identical tree, so row 3's green was about the declaration and never about the diff.
Correction to one inherited reading, re-derived from the file rather than the card: the card and its source comment both say the
Check Changesetjob "skips label events". On this base it does not — onlyCheck PR SizeandAuto Labelcarry that exclusion. Nothing in defect B depends on it (there was still noeditedtrigger), but the sentence should not be inherited.Defect A — what changed, and why it had to be the conclusion
NOT MEASUREDwas one verdict at exit 0, so the check run concludedsuccesswhether the reading had passed or had never happened. It is now split by the only question that matters — could the missing reading have changed the answer?not-measured-moot— no changeset gradespatcha package whosepackages/*/src/**this diff moves, soyesandnoreach the same verdict. Exit 0, and the line now says why it is green rather than printing a tick that means nothing.not-measured-material— apatchsits on a package this diff grew, so the declaration is the difference betweencleanandenforce, and it was not readable. Exit 1, naming the changeset, the package, and both declarations.⛔ No tolerance, no allowlist, no comment-only heuristic, and nothing was weakened to make a case go green: the explicit
Clause-②: noremains a pass on the very tree the unread reading refuses, and every previously-refusing input still refuses.Why the conclusion route and not the output/summary route. Measured, not assumed, on a real run of this job (
Check Changeset, job 101999797457, 2026-09-08): the check run'soutput.titleisnulland itsoutput.summaryis the empty string. A GitHub Actions job publishes no check-run output of its own, so "put it in the summary" is not a cheaper spelling of the same fix — it is a request for a second mechanism (achecks: writeAPI call creating a check run) whose only readers would be human.Check Changesetis not in that set: read live from ruleset 12119582 (repository-sourced,active,includes_parents=truereturns only it), the required contexts areTypeScript Type Check,Test Core,Dogfood Regression Gate,Build Core,Temporal Conformance (live PG + MySQL),Lint and Repo Gates,Governed Surface Queue Guard. So this gate is advisory at the ruleset layer — and that makes the conclusion route stronger, not weaker: the conclusion is the ONLY authority this gate has (the red X,gh pr checks, every summary view), and it is the same authority through which its "you forgot a changeset" verdict is enforced today. A step log is one layer below the only surface that carries this gate at all. That the set does not include it is recorded in the acceptance notes below, not fixed here.Defect B —
edited, and what it costsThe remedy for the new refusal is a PR-body edit, and a
pull_requestpayload is a snapshot thatrerun_failed_jobsreplays. Without aneditedtrigger the red could not be cleared by anything short of pushing a commit — the permanently-red-by-construction shape 5580 and 6378 exist to remove. So the two halves of this card are one change: A's refusal is unshippable without B.pr-automation.ymlnow subscribes toedited, exactly as this repo's two other PR-body-scoped blocking checks (duplicate-fix-guard.yml,partof-closing-keyword-guard.yml) already do, and for the argument their headers already carry.Added run volume, measured (2026-09-01 to 2026-09-08, this repo's own run counts through
GET /actions/workflows/FILE/runs?event=pull_request):check-links.yml[opened, synchronize, reopened]duplicate-fix-guard.yml+ editedpartof-closing-keyword-guard.yml+ editedpr-automation.yml+ labeled, unlabeledThe two
editedreadings agree exactly, which is the control on the arithmetic; a paths-filtered workflow over the same window returns 1363, which is the control that the endpoint is not answering with a constant. Soeditedis about 557 events / 8 days, ~70 a day, about +15% on this workflow's own 3684 — and it buys one job, not three:Check PR SizeandAuto Labelare excluded fromeditedby the same sentence they already carry for label events (a title or body edit moves no file), which also keeps their behaviour byte-identical to before. OneCheck Changesetjob is ~46s (measured on the same job above).github.event.changes.body. It would trim title-only edits, and its failure direction is this defect returning silently: an expression that misjudgeschanges.bodyon an empty previous body stops the gate re-reading and nothing says so. 70 runs a day does not buy that risk.Blast radius, measured on the population rather than argued
Every one of the 45 most recent merges to
main, judged with the real script against each PR's real body and labels, before and after this change (31 judged, 14 exempt byskip-changeset):yesstill reachesclean), so they stay green and now say why.successover@objectstack/spec: patchbeside a 124-line move inpackages/spec/src/api/rest-server.zod.ts. Its author had done the thinking and written it down; the gate could not read it and said so where nobody with merge authority looks. That is this card, live, after the fact.The whole population is its own control: every other row is byte-identical before and after.
Two lanes that must NOT be able to red, and are pinned so
cut-rc.ymlruns this same script over a whole RC snapshot range on aworkflow_dispatch, where there is no pull request and no declaration to read. A rule that reddened there would be unshippable, so the absence of apull_requestpayload is its own verdict (no-pull-request, exit 0) rather than an unread declaration — and the same absence ON apull_requestrun is a failure instead, because the runner writes that file. The self-test pins both directions, pins that nocut-rc.ymlcall site grows an--event, and pins thatcut-rc.ymlstays offpull_requesttriggers.The stand-down lane needs a positive
payload: falsefrom the reader: a caller that omits the flag falls through to the lanes that can refuse. Unknown provenance enforces.Verification
node scripts/check-changeset-no-major.mjs --self-test— 179 assertions, exit 0. The LEVEL battery grew 41 to 56 cases and the wiring battery 15 to 22; both floors ratcheted to the new counts.46edc93a6) and green again.editedremoved from the trigger list, same proof): the wiring assertion reds by name, quoting the trigger list it read. Restore verified byte-identical (blob6f349934a) and green again.node --stack-size=4000 node_modules/eslint/bin/eslint.js . --no-inline-configover 6376 files, 0 errors, 0 warnings, exit 0, at commit675e3d0a50.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackfrom the working change set: 50 commands, 49 green. The one exception ispnpm check:pm-dispatch-gates, which exceeded the container's foreground limit twice while a sibling agent held the same gate; it is reported as NOT MEASURED rather than as a pass or a failure, and CI runs it.skip-changesetinstead, measured rather than assumed: all 23 published packages shipdist,README.mdandCHANGELOG.mdand nothing else (positive control:distpresent in 23 of 23), so nofiles[]entry can carry.github/**orscripts/**.What is deliberately NOT here
The card's second direction for defect B — requiring the
Clause-②:line in the PR body from the first push — is partly a change to os-dev's own contract in.claude/agents/os-dev.md. That path is a governed surface (docs/adr/** · .claude/** · skills/** · AGENTS.md · CLAUDE.md), so it is not in this diff and is not this seat's to land. The gate half stands on its own and does not depend on it: this PR refuses only where the missing declaration decides the verdict, which is a much narrower demand than "every PR must declare".验收备注
Check Changesetis not in the repository's required-status-check set (measured above). A gate that carries a maintainer ruling being advisory at the ruleset layer is a fact worth someone's decision, and it is a repository setting no PR can change — noted for the maintainer, not filed and not fixed here.editedtrigger fires on title edits too. Left unfiltered on purpose, with the argument and the volume above.allow-majorskips the whole guard step, this axis included. Unchanged, and already recorded in the script's own header.Generated by Claude Code